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Executive Summary 



As current educational reform efforts seek to ensure accountability for 
all of our nation’s students, increasing interest is being placed on the 
extent to which current assessment practices and programs include stu- 
dents with disabilities. Developing accurate reporting procedures on 
the participation of students with disabilities in large-scale assessment 
programs has proven to be difficult, due to a lack of data, differing def- 
initions of eligible testing populations, and the misalignment of data 
collection efforts and data management responsibilities. Specific rec- 
ommendations are provided for both policymakers and local practi- 
tioners and administrators to assist in improving our ability to accu- 
rately report the participation of students with disabilities in statewide 
assessment programs. These include suggestions to: 

• Identify students with disabilities in statewide assessment 
programs 

• Standardize procedures for calculating participation rates 

• Improve communication between state special education and 
assessment offices 

• Ask good questions about state and district assessment 
practices 

• Help parents understand the importance of participating in 
assessment programs 

• Evaluate current district and state policies on participation 
and accommodations in assessment programs 
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Assessment: A Cornerstone of Reform 



For many school districts and state departments of education, the 1990s 
have heralded dramatic and fundamental change. Local and state educa- 
tion agencies have adopted diverse strategies in redefining the expecta- 
tions of their educational systems, and have put forth myriad conceptual 
frameworks of goals, outcomes and content standards. The attention of 
many policymakers has already begun to turn from discussions about what 
students should know and be able to do, to questions about how best to 
measure the extent to which students attain these competencies. 
Consequently, statewide assessment systems, using both traditional and 
new methodologies (e.g., performance assessments and portfolios) have 
emerged as critically important components of this reform movement. 

Nearly every piece of federal and state legislation focused on educational 
reform contains inclusionary language; that is, standards, goals, assess- 
ments and accountability systems are touted as being for all students. All 
students are to work toward attaining high standards, and all are to have 
access to learning opportunities that will enable them to attain those stan- 
dards. Schools, school districts and states are to report on the progress of 
all their students, and thus all students are to participate in assessment pro- 
grams. Consequently, LEAs and SEAs have been making efforts in recent 
years to have all students participate in assessments and to report on the 
numbers or proportions who do so. To the extent that there is variability 
among states in participation rates, differential exclusion from assessment, 
or inaccurate reporting, state and district comparisons are invalid, and pol- 
icy decisions based on such results remain questionable. 

In this paper we examine the present variability in the way states report the 
participation of students with disabilities in their testing programs. Many 
of the conclusions drawn in this report have emerged from research con- 
ducted by the National Center on Educational Outcomes (NCEO) at the 
University of Minnesota. The primary mission of NCEO has been to work 
with federal and state agencies to facilitate and enrich the development and 
use of indicators of educational outcomes for students with disabilities. To 
accomplish this mission, NCEO has examined current educational assess- 

O C E o T 
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merit practices to determine the participation rates of students with dis- 
abilities in national and state assessments, and the current accessibility and 
use of data on the results of education for students with disabilities. 

Will “All” Ever Mean “AH” in Assessment? i 

An early analysis of nine major national data collection programs by 
NCEO researchers (McGrew, Thurlow, Shriner & Spiegel, 1992) revealed 
that between 40 and 50 percent of school-aged students with disabilities 
were being excluded from these nationwide efforts. In large scale assess- 
ments such as the NAEP Trial State Assessment Program of 1990, exclu- 
sion rates among the participating states ranged from 33% to 87% of stu- 
dents with disabilities. Such variability prohibits valid comparisons 
between states, and prevents policy-relevant findings to be drawn about 
how students with disabilities are benefiting from their educational expe- 
riences. 



Students with disabilities have been excluded from achievement testing 
programs for several reasons. One primary motivation for schools and dis- 
tricts to exclude them is the existence of high stakes statewide account- 
ability systems that compare the performance of schools and districts, and 
often make awards or sanctions based on these results. Schools within 
such systems are motivated to minimize the number of low-performing 
test takers in order to raise their overall test scores. Compounding this 
problem is the variability in state and district policies about who gets test- 
ed. Zlatos (1994) examined 14 major urban school districts and found that 
participation rates in testing varied from 93% of all enrolled students in 
Memphis to 66% in Boston. This system of academic “red shirting” of stu- 
dents with learning problems has resulted in a system that perpetuates 
invalid comparisons among our nation’s schools and school districts. 



Other reasons for exclusion from testing stem from the fact that many of 
the standardized tests currently in use were not originally designed or 
normed with any consideration of students with disabilities. National 
norms often were established on only samples of students without disabil- 
ities. Furthermore, there have been few studies of the effects of various 
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testing accommodations on the validity or reliability of testing instru- 
ments. Only recently have federal funds been directed toward research on 
these issues. The U.S. Office of Educational Research and Improvement 
has funded eight states (and one consortium of states) to investigate issues 
related to standards and assessments, particularly focusing on the effects 
of using accommodations for students with disabilities and limited English 
proficiency on the technical integrity of assessment instruments. 

Students with Disabilities in Assessment: An Emerging Issue 

New federal policies have placed pressure on states to ensure the partici- 
pation of students with disabilities in their overall assessment systems. 
Language within Goals 2000: Educate America Act (Public Law 103-227) 
clearly includes students with special learning needs in its mandate for 
states to set high standards for all students. Amendments drafted for the 
reauthorization of IDEA (Individuals with Disabilities Education Act) by 
both the House and Senate specifically mandate the inclusion of students 
with disabilities in statewide assessment systems, preferably through reg- 
ular assessment systems or through alternative means for those who can- 
not participate in the regular testing program. Other pending language 
within Part B of the Act stipulates that states will report on the academic 
performance of students with disabilities in the same way and with the 
same regularity as they do for students without disabilities. 

This federal commitment to a fully unified system of educational account- 
ability has created unique challenges for state and local policymakers and 
assessment experts alike. Because of this, in part, much attention is now 
being given to examining current testing policies and their impact on stu- 
dents with disabilities. In a recent literature review of state level policies 
on testing participation and accommodations, NCEO staff found that near- 
ly all states have revised such policies within the past two years (Thurlow, 
Scott, & Ysseldyke, 1995a, 1995b). 
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survey of state 
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found that state 
special education 
directors could 
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rates for only 49 of 
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the total number 
of tests in use. 
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would seem to 
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particular age or 
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What We Know (and Don’t Know) about Participation Rates 

Since 1991, NCEO has conducted an annual survey of state directors of 
special education regarding educational outcomes for students with dis- 
abilities. Specific survey questions have solicited information on the par- 
ticipation rates of students with disabilities in statewide achievement test- 
ing. Findings over the past four years have indicated that the participation 
of students with disabilities in statewide assessments continues to vary 
considerably from state to state, with estimates, when given, ranging from 
zero to 100%. Furthermore, respondents are unable to provide estimates 
of participation for the great majority of the assessments presently admin- 
istered. In its 1994 survey of state practices, NCEO found that state spe- 
cial education directors could report participation rates for only 49 of the 
133 assessments used during that year, less than 37% of the total number 
of tests in use (Erickson, Thurlow, & Thor, 1995). Since respondents were 
allowed to give approximations, it is not known how many of the report- 
ed rates were actually calculated from verifiable data sources. 

What is known is how controversial such data can be. Even though NCEO 
has consistently verified responses before publishing its annual survey, 
each year has brought a new round of questions about the accuracy of the 
participation rates being reported. Usually, those questioning the report’s 
accuracy believe the published rates are too conservative; on occasion, 
however, some have insisted the estimates are inflated. Most questions are 
raised by state assessment officials whose offices may or may not have 
been consulted by the state special education units submitting these esti- 
mates. 

Why are Participation Rates So Elusive? 

On the surface, calculating participation rates for students with disabilities 
in statewide assessment programs would seem to be a relatively straight- 
forward task of dividing one number by another: the number of students 
with disabilities who take the test, divided by the population of all students 
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with disabilities at the particular age or grade level being tested. Yet 
states report considerable difficulty when locating accurate data with 
which to build this deceptively simple ratio. Reported problems general- 
ly fall into three categories: (1) lack of data on those students taking the 
test; (2) differences in determining eligible testing populations; and (3) 
misaligned data collection procedures. These issues are addressed in the 
following sections. 

Neglected Numerators: The Problem of Simply Not Knowing 

Participation rates obviously cannot be calculated without first knowing 
who participated in the actual assessment. Regrettably, this basic infor- 
mation is not routinely collected in the testing programs of many states. In 
its 1995 survey of state directors of special education in the 50 regular and 
10 unique states, personnel in 12 responding states did not know whether 
students with disabilities could be identified within their states’ assessment 
databases, and those in another 20 states reported that they definitely could 
not (Erickson, Thurlow, Seyfarth & Thor, 1996). To further complicate 
this matter, testing administrators or classroom monitors may not always 
know which tested students are receiving special education services, and 
therefore cannot provide accurate documentation at the time of testing. 

Drifting Denominators: The Problem of Eligible Populations 

Variability also occurs in the way educators define the eligible population 
against which to compare the number of test takers. Presently, the denom- 
inator of any participation ratio may be calculated in different ways. With 
so many different policies in place to determine whether a student with 
disabilities will participate in any particular assessment, no standard 
method exists to report on which such students are being included. In 
Figure 1, the number of students found in the inner circle would be the 
numerator in calculating a participation rate. All students with disabilities 
at the particular age or grade level being tested (i.e., the biggest circle) 
would constitute the denominator. However, current assessment policies 
often exclude certain subgroups of students with disabilities and leave 
them literally “out of the equation.” Consider the following scenario: 
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Current 
assessment 
policies often 
exclude certain 
subgroups of 
students with 
disabilities, thus 
leaving them 
literally “out of the 
equation.” 



All students 

with disabilities 
at the age or 
grade level 
being tested 



All students _ 
with disabilities 
considered 
"eligible" for 
participation in 
assessment 

Figure 1 . Model for Determining Participation Rates 

An education official is asked for information on the percentage of 
students with disabilities who participated in a statewide fourth 
grade reading assessment. Checking the aggregated results, the 
official discovers that 5000 students participating in the examina- 
tion were coded as having an Individualized Education Program 

> 

(lEP). This number becomes the numerator. Now the official’s 
attention turns toward determining the denominator, that is, the 
population of students with disabilities against which this numera- 
tor will be compared. He knows that the state assessment program 
does not test those students with disabilities attending separate 
schools, private schools, residential programs, correctional facili- 
ties, or those receiving homebound services. These placements 
account for approximately 10% of all fourth grade students receiv- 
ing special education services. With this in mind, he finds that on 
a statewide basis, 7500 students with disabilities were at the age 
equivalent of fourth grade. Reducing this figure by 10%, or 750 
students, the assessment official calculates a participation rate of 
74% for students with disabilities in the fourth grade reading 
assessment (i.e., 5000 -r 6750). But in fact, only 67% of students 
with disabilities at the fourth grade level were assessed (i.e., 5000 
7500). 




All students 
with disabilities 
participating in 
assessment 
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As this example shows, the road to participation in assessment seems to 
have more than its share of wayside rests for students with disabilities. 
Besides the primary educational setting (e.g., special school, residential 
facility, ungraded program, or special classroom) several other factors are 
often considered in determining assessment eligibility, including: 

• A student’s disability category; 



• The extent of a student’s access to the general curriculum. In the 
past, for example, NAEP participation criteria allowed for the exclu- 
sion of a student with an lEP if that student was mainstreamed “less 
than 50 percent of the time in academic subjects and is judged to be 
incapable of taking part meaningfully in the assessment’’ (Mullis, 
1990). At the state level, the following true example is typical of 
current policy: 

A state official reports a participation rate of 77% for 
all "eligible” students with disabilities in a statewide 
achievement testing program. Testing eligibility is 
defined as having an educational program that is "aca- 
demically focused. ” The official also reports that only 
60% of all students with disabilities in the state are con- 
sidered eligible for participation in testing because of 
this reason. What then is the true participation rate — 

77% or 46% (i.e., 77% of 60%)? 



The road to 
participation in 
assessment 
seems to have 
more than 
its share of 
wayside rests 
for students 
with disabilities. 



• Completely individualized reasons, through the decisions of indi- 
vidual administrators or teams making decisions about testing stu- 
dents with disabilities. These decisions may or may not be docu- 
mented on the students’ lEPs. 



These various points of exclusion make the interpretation of participation 
rates between schools, districts or states very problematic, since the report- 
ed rates of participation may be excluding many special education students 
from the population being referenced. 



OEO 

ERIC 



7 



. A 



Fractured Fractions: Problems of Timing and Responsibility 

Another issue surrounding the accurate determination of participation rate 
is one of differing data collection cycles, that is, the time at which data 
reflecting the numerator and denominator are collected. States are 
required to submit special education child count data to the federal gov- 
ernment on December 1st of each school year. The testing cycles of many 
states are in the fall or spring. Under such conditions, even the most accu- 
rate data collection can produce unusual results, including participation 
rates that exceed 100%. The problem is easier to explain than it is to fix: 
a December 1 population count does not reflect those students becoming 
eligible for special education between December and a later testing cycle. 
Nor does it reflect students who discontinue services. Therefore, partici- 
pation rates calculated using less than current population information can 
result in estimates that largely exaggerate or underestimate the true rate at 
which students with disabilities are participating. 

Participation ratios are “fractured” not only by time, but by departmental- 
ized responsibilities in data collection. In many state departments of edu- 
cation, responsibility over large-scale assessment data falls to designated 
officials within assessment or evaluation divisions, while information on 
students with disabilities remains in the domain of state special education 
offices. Much of the confusion about the determination of participation 
rates is a direct reflection of poor inter-departmental communication 
between these units. As one state assessment official recently stated, "We 
provide the numerator, and the special education division provides the 
denominator." 

Recommendations for Policymakers 

Should the amendments currently being considered for the reauthorization 
of IDEA become law, states will be asked to report the participation rates 
of students with disabilities as part of their annual state reporting proce- 
dures. If this becomes the case, states will undoubtedly seek direction 
from federal policymakers about which students may or may not be con- 
sidered part of the population against which test takers with disabilities 



should be compared. Three general recommendations can be made based 
on NCEO’s experience in pursuing accurate participation data from 
statewide assessment programs: ^ 

Identify students with disabilities in statewide assessment programs. 
Determining an accurate numerator is problematic in many states, with 
officials unable to determine how many test takers were actually students 
with disabilities. NCEO suggests that efforts be undertaken immediately 
within all state assessment programs to identify those test takers who were 
being provided special education services at the time of the test. This vari- 
able could readily be added to the other demographic descriptors routine- 
ly collected by state assessment programs. 

Successful implementation of this reconmiendation necessarily would 
require cooperation between local testing administrators and special edu- 
cation personnel. Because general education teachers may not always be 
aware that a student is receiving special education services (particularly 
indirect monitoring or consultation services) it would be essential for spe- 
cial education personnel to verify all assessment rosters. 

Standardize procedures for calculating participation rates. Until clear 
direction is given to state assessment programs and special education 
offices, the problems surrounding variance in determining a suitable 
denominator for calculating participation rates will continue to plague all 
attempts to track increased participation of students with disabilities in 
statewide assessment programs. Efforts should be undertaken by policy- 
makers at the state and federal levels to provide clear directives in how 
such numbers should be derived. Because of their inclusiveness, 
December 1st child counts of students with disabilities (reported to the 
U.S. Department of Education) currently are our best available source for 
determining a reasonable denominator in the participation ratio. Because 
this count includes a state’s entire population of children receiving special 
education services at a set date, states would have a standardized and 
widely recognized metric to use when calculating rates of assessment par- 
ticipation. 



Identify students 
with disabilities 
in statewide 
assessment 
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rates. 
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special education 
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offices. 
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As this paper has noted, however, the December 1st child count is not 
without its limitations. New means should be explored to assist states in 
bringing their special education population counts and testing cycles into 
alignment. And because the child count reports data by age level, age-to- 
grade conversions should be established and publicized for consistent 
interpretations of which students should be included in the denominator of 
participation ratios. 

Improve lines of communication between state special education and 
assessment offices. If accurate reporting of participation by students with 
disabilities in statewide testing is going to happen, a collaborative strategy 
will need to be developed by these two educational units. The many issues 
surrounding the status of students with disabilities in testing programs 
need to be cooperatively identified and resolved. Improved levels of inter- 
action between assessment and special education offices would undoubt- 
edly lead to improved confidence in our methods of measuring and report- 
ing the participation of students with disabilities in statewide assessment 
programs. 

Recommendations for Practitioners 

What can teachers and local special education administrators do to 
improve our ability to report on the participation of students with disabil- 
ities in statewide assessment programs? The NCEO suggests a number of 
ways in which positive change can be encouraged at the school or district 
level: 

Ask some good questions. Are students with disabilities within your 
school or district included in assessment programs? Are students with dis- 
abilities being held to the same academic standards and expectations as 
those of general education students? Is your school accountable for all its 
students? Posing these questions uncovers the fundamental issues sur- 
rounding our own expectations for students with disabilities, and their 
access to the curriculum provided to their non-disabled peers. 
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Help parents make the connection. Many parents of students with dis- 
abilities are likely to feel that their children have already been tested 
enough. But the majority of assessments given to students with disabili- 
ties involve determining their eligibility for services, not measuring their 
educational progress. Parents can play a pivotal role in advocating for the 
inclusion of students with special needs in assessment, but only if they are 
provided information on how participation in testing can lead to higher 
expectations, broader curricular offerings, and ultimately, better results for 
students. 

Critique your assessment policies. Most states and many individual dis- 
tricts have policies in place that oversee the participation of students with 
disabilities. Ask to see them. Do they promote the inclusion of students 
receiving special education? Are appropriate accommodations allowed for 
use by students who need them? Are the testing results for students with 
disabilities included when reporting on the performance of schools or dis- 
tricts? Are the results used by special education administrators for plan- 
ning improvement efforts? If you think such policies need improvement, 
bring your concerns to testing officials and local school administrators. 



In Conclusion 

Is this much ado about nothing? We think not. Failure to include students 
with disabilities in assessment and accountability systems leads to failure 
to assume responsibility for the results of their education. Partial partici- 
pation of a state’s or district’s students in assessment can result in policy 
decisions being made on partial or skewed data. Inaccurate deriving or 
reporting of participation rates provides policymakers with inaccurate 
comparisons among states, districts, cities, schools, or even between stu- 
dents with disabilities and their peers without disabilities. Our national, 
state and district educational policies should be based on complete sets of 
data on all of America’s school children. 
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